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DESCRIPTION 

MPEG-21 DIGITAL CONTENT PROTECTION 
SYSTEN 



5 TECHNICAL FIELD 

The present invention relates to Digital Rights Management (DRM) or 
Intellectual Property Management and Protection (IPMP) for a generic digital 
content, especially relates to the protection and management of a digital content 
independent of any data format. 

10 

BACKGROUND ART 

As various kinds of network are widely deployed, it will be demanded that 
digital content can be delivered and distributed to user via such network besides 
using CD, DVD. The corresponding issue is raised by content owner. Is it secure to 
15 sell their content in this way? 

As hard disk or other storage embedded device become more and more, 
another issue is that how the content protection technique can ensure the entitled 
rights to be exercised correctly. 

As many different digital formats exist to use for packaging content in 
20 digital form for easy transmitting over various network, question arises as how 
the protection technology can be cross-used among different digital formats. 

At the same time users have more demands on the convenience with low 
cost for enjoying content, even sharing with their friends if they purchase such 
rights, to have rich user experience. 
25 Conflict is always there since content owner cares for any illegal copy so 

that content providers are trying to protect content in their own proprietary ways 
due to lacking of the open protection techniques in the market at that time. 
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This not only brings a big barrier for content owner to sell content, but also brings 
a heavy cost for CE (consumer electronics) manufacturers to produce different 
versions of the product just for matching with various protection techniques which 
5 content provider use . 

MPEG-21 is trying to define a generic framework to enable transparent 
and augmented use of digital content across a wide range of networks and 
devices used by different communities. How to protect the contents when they 
are being used across network or devices, becomes a very important item in 

10 MPEG-21, which is the part 4 of MPEG-21, called MPEG-21 IPMP (Intellectual 
Property Management and Protection) 

In the past, people working on MPEG-4/2 IPMP Extension were 
required to define a content protection scheme based on MPEG-4/2 system since 
the aim is to protect any content if they are packaged in MPEG-4/2 format. 

15 In MPEG-21, a Digital Item (DI) is defined as a structured digital object 

for any digital content with a standard representation, identification and 
description, and it will be used as the fundamental unit of interchange, 
distribution and transaction within MPEG-21 framework. 

The Digital Item is declared and expressed using XML by Digital Item 

20 Declaration (DID). Besides a digital content which is represented as media 
resources in MPEG-21, such as video, music, image, the DID provides the 
flexible structure to include various kinds of functional metadata. Such 
metadata is supposed to describe media resource format, to specify resource 
protection scheme, to give the resource an identification name, to provide User 

25 preference, etc. 

Besides the core part of DID technology, some other key technologies 
have also been elaborately developed or are under development. Digital Item 
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Identification (DII), Digital Item Adaptation (DIA), Intellectual Property 
Management and Protection (IPMP), REL (Rights Expression Language)/RDD 
(Rights Data Dictionary), as well as ER (Event Reporting) are all the important 
technologies for extensively exploiting the Digital Items' usage. All the 
5 functional metadata defined by these technologies can be placed into a DID 
document to aid the actual media resource consumption. 

A content protection and management mechanism is highly requested to 
address most of the requirements raised by many different application domains, 
especially in the scope of MPEG-21 domain, to reflect the market needs. 
10 The requirements on MPEG-21 IPMP are the problems to be targeted 

and solved here. 

IPMP, especially MPEG-21 IPMP shall support the management and 
protection of intellectual property in descriptors and description schemes. 

IPMP, especially MPEG-21 IPMP shall provide for interoperability so 
15 that content is able to be played anywhere. 

IPMP, especially MPEG-21 IPMP should enable devices to dynamically 
discover, request, and obtain upgrades for supporting new media formats, IPMP 
tools and support. 

IPMP, especially MPEG-21 IPMP shall provide mechanisms to reference 
20 Digital Item Descriptions as part of the language, make reference to external 
content descriptions. 

IPMP, especially MPEG-21 IPMP shall provide mechanisms to associate 
Expressions with composite Digital Items. 

IPMP, especially MPEG-21 IPMP shall provide mechanisms to reference 
25 Containers or other aggregations of Digital Items. 

IPMP, especially MPEG-21 IPMP should flag that a particular 
Expression should be subject to protection. The protection itself (if any) is 
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provided by an IPMP system controlling the Expression as a Digital Item. 

IPMP, especially MPEG-21 IPMP shall provide mechanisms to reference 
authentication schemes. 

IPMP, especially MPEG-21 IPMP shall provide mechanisms to ensure 
5 that the IPMP is independent of the format or delivery channel of Digital Items. 

IPMP, especially MPEG-21 IPMP shall unambiguously articulate 
requirements relating to D?MP Tbol and Features. 

IPMP, especially MPEG-21 IPMP shall need to identify IPMP Ibols and 
Features to build trusted IPMP implementations. 
10 IPMP Ibols and Features are components parts to build an IPMP- 

enabled Terminal or Peer. It should also possible for a Terminal or Peer to 
disclose its IPMP capability (IPMP Tools and Features). This makes it possible 
for a communicating Terminal or Peer to examine IPMP capability of another 
Terminal or Peer before deciding to engage with it. 

15 

DISCLOSURE OF THE INVENTION 

Methods of Digital Content Protection with Digital Rights Expression, 
comprising the following steps of: 

Parsing a digital content description, especially parsing a DID (Digital 
20 Item Declaration) in MPEG-21 scope; 

Retrieving a digital content identifier (content ID) which is used to 
identify the said digital content, especially a DII in MPEG-21 scope, or sub 
content identifier; 

Detecting a rights and protection description holder which contains 
25 rights and protection information applied to the said digital content with the 
corresponding content ID, and here the holder called IPMP (Intellectual 
Property Management and Protection) Control Graph Holder or EEL (Rights 
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Expression Language)-IPMP Control Graph holder; 

Retrieving a flag from the said holder which indicates if the said content 
is protected or belongs to free content; 

Processing the description information carried in the said IPMP Control 
5 Graph or REL-IPMP Control Graph; 

Checking if rights descriptions or other metadata description is digital 
signed by retrieving a flag which is attached to the said rights or other 
metadata; and if it is signed, preparing the corresponding digital signing tool 
which is indicated by IbolID; 
10 Retrieving a key license from a protected License Manager; 

Checking the integrity of the said rights or metadata using the said 
digital signing tool; 

Parsing the said rights with their conditions following the rules which is 
pre-defined, especially following REL rides which is defined in MPEG-21 scope, 
15 and storing the said entitled rights and conditions in a buffer for future 
checking; 

Checking if the said content is encrypted by retrieving a flag which is 
attached to the said content; and if it is encrypted, preparing the corresponding 
encryption tool which is indicated by ToolID; 
20 Un-protecting the said encrypted content using the said encryption tool 

with the said IbolID, and other information; 

Checking if the said content is watermarked by retrieving a flag which is 
attached to the said content; and if it is watermarked, preparing the 
corresponding watermarking tool which is indicated by IbolID for further 
25 action; 

Processing user's request against the said entitled rights and conditions 
stored in the buffer; 
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Exercising the rights requested by the said user if it is entitled, and 
Acting on the said un-protected content for playing, rendering, recording, 
modifying, deleting, adapting, etc. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 shows a Prior Art: DID Structure with Possible Protection 
Information Included. 

Figure 2 shows a Prior Art: MPEG-21 IPMP Architecture. 

Figure 3 shows Content Packaging Flow Chart with separate Rights & 
10 Protection. 

Figure 4 shows Terminal Processing Flow Chart for Protected & 
Packaged Content with IPMP Control Graph Information. 

Figure 5 shows Content Packaging Flow Chart with mixed Rights & 
Protection. 

15 Figure 6 shows Terminal Processing Flow Chart for Protected & 

Packaged Content with REL-IPMP Control Graph Information. 

Figure 7 shows IPMP Control Graph for Rights & Protection 
Information Carried in DID. 

Figure 8 shows IPMP Architecture with IPMP Control Graph Processed. 
20 Figure 9 shows IPMP Architecture with REL-IPMP Control Graph 

Processed. 

Figure 10 shows Layout of Rights and Protection in REL-IPMP Control 

Graph. 



25 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
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(Means of Solving the Problems) 

On the content packaging side: 

By introducing the concept of IPMP Control Graph to refer to all the 
rights and protection information which is directly associated with the content 
5 By defining IPMP Control Graph or REL-IPMP Control Graph as 

protection metadata holder to contain rights and protection information which is 
used to package and protect the content; 

By placing rights & condition in the IPMP Control Graph or REL-IPMP 
Control Graph; 

10 By placing content encryption information in the IPMP Control Graph or 

REL-IPMP Control Graph; 

By placing watermarking information in the IPMP Control Graph or 
REL-IPMP Control Graph; 

By placing rights protection information in the IPMP Control Graph or 
15 REL-IPMP Control Graph; 

By placing and indicating key information which is used to encrypt 
content in the IPMP Control Graph or REL-IPMP Control Graph; 

By placing key/license information in the IPMP Control Graph or REL- 
IPMP Control Graph, or in Rights, DID, or somewhere indicated by keyLocation; 
20 By indicating which IPMP Tool is used for encryption, digital signing, 

watermarking with ToolID in the IPMP Control Graph or REL-IPMP Control 
Graph; 

By associating rights and protection with the protected digital content or 
its sub content vising content ID or DIE and sub content ID; 
25 By placing IPMP Control Graph or REL-IPMP Control Graph in DID 

container or other appropriate place in other application domains; 
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On the terminal side: 

By parsing DID to retrieve content ID or sub content ID, and IPMP 
Control Graph or REL-IPMP Control Graph; 

By parsing IPMP Control Graph or REL-IPMP Control Graph to retrieve 
5 Rights and Protection related descriptions; 

By invoking IPMP tools which are used to protect the content or rights, or 
other metadata; 

By retrieving key information from KeyData Holder directly of indirectly; 
By retrieving a key license from a protected License Manager; 
10 By un-protecting the protected content using the above obtained 

information; 

By checking Rights' integrity using the tool indicated by ToolID; 
By parsing the rights and conditions which are embedded with the 
content; 

15 By retrieving watermarking descriptions and preparing for further 

action; 

(Operation of the Invention) 

On the content production side as shown in Figure 3, IPMP Control 

Graph is generated as shown in Figure 7, to contain all the rights and protection 
20 information which is directly associated with the content identified by content 

Identifier (CID) or DII if MPEG-21 could be used. 

The content could be watermarked using certain watermarking tool to 

achieve certain functions, such as finger printing, persistent association, or 

copyright protection by embedding CID or other information. 
25 The content can be encrypted by an IPMP tool with TbolIDXXX, where 

xxx is the number which is registered with RA (Registration Authority), to 

indicate which encryption algorithm is used. A default tool such as AES is 
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defined for simple hardware to implement. The resulted Key information could 
be carried in IPMP Control Graph directly or by pointing to a location where the 
whole Key information data could be found. The encryption key can be further 
encrypted and finally a license could be generated and directly carried in either 
5 IPMP Control Graph, in BEL data or other Rights Expression Data, or in DID 
itself, or in somewhere which can be indicated by KeyLocation indicator to be 
carried in IPMP Control Graph/REL/DID; 

However the segments of key information would also possibly be 
packaged together with the associated content segments when the protected 
10 content is transmitted via network for synchronization purpose. 

Rights can be expressed by an independent and existing technology 
standard such as REL defined in MPEG-21 or other Rights Expression methods, 
and such rights could be protected by digital signature for its integrity; 

On the content consumption side as shown in Figure 4, a packaged 
15 content with rights and protection information is subjected to IPMP Control 
Graph parsing, from there it can be known if the content is protected and 
furthermore to determine whether the content is encrypted, watermarked, or 
rights is protected as well; 

The corresponding protection tools would be invoked and acted on the 
20 protected object, the tools can be those normative tools defined by MPEG-21 
standard and hence installed in the device, or the tools can be proprietary and 
identified by tool IDs which can be downloaded from a remote location; 

Tbol is identified by a registered Tbol ID, which is a flag to tell terminal 
or device to prepare the corresponding tool or locate the tool beforehand; 
25 The key information is retrieved from KeyData Holder defined and 

carried in IPMP Control Graph directly or indirectly, and it would also possibly 
be obtained in segment with the corresponding content segment to be protected 



WO 2005/036882 



10 



PCT/JP2003/013120 



if the content is distributed through network. 

The license information can be obtained from License Manager which 
could be a temper resistant entity to prevent any disclosure of how a license is 
retrieved by a license manager. 
5 Rights and content is un-protected by using the above key, key data, and 

protection tool. Rights is further parsed by Rights Parser to obtain the rights 
and conditions in clear form, so that the rights and conditions processing can be 
conducted. 

Therefore the un-protected content can be played back, rendered, 
10 modified, deleted, or adapted if there is such rights entitled for the user; 

(EXEMPLARY EMBODIMENT) 

As shown in Figure 1 for the prior art [see reference 1 and 2], a digital 
content is packaged by DID with possible protection associated. 
15 (REFERENCE 1, 2) 

[1] "ISO/IEC 21000-2 MPEG-21 Digital Item Declaration FDIS", ISO/IEC JTC1 
SC29/WG11/N4813, May 2002 

[2] Patent on "Apparatus of a MPEG-21 System", inventors: Zhongyang Huang, 
Ming Ji, Shengmei Shen, Taka Senoh, Takuyo Kogure, Takafumi Ueno with 

20 internal patent number Pat01.028, filed in Japan in Feb.2002. 

The DID has defined a useful model (unit 1.1 in Figure 1) formed by a 
set of abstract terms and concepts such as Container, Item, Component, Anchor, 
Descriptor, Condition, Choice, Selection, Annotation, Assertion, Resource, 
Fragment, Statement, etc (e.g. shown in Figure 1 unit 1.6, 1.7, 1.8) for defining 

25 Digital Items. 

Module 1.2 shown in Figure 1 is the overall IPMP Control Information 
used for all the items to be protected inside this container. Module 1.3 and 1.4 
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are the specific protection information which is associated to the protected 
content. Module 1.5 is the DII to indicate the content ID. 
The further improvements over the Prior Art are: 

Since DID is to address static relation among each elements and it can 
5 be treated as file format, rights and protection information can be directly 
associated to its protected content as IPMP_Control_Graph, shown in Figure 3. 

On the other hand, key information can be carried from KeyData Holder 
in IPMP_Control_Graph directly or indirectly. It could also be segmented when 
the content is delivered via network. 
10 Rights which might be encrypted is carried separately or together with 

protection information. 

Another Prior Art is shown in Figure 2 [see reference 3] for MPEG-21 
IPMP Architecture. 

(REFERENCE 3) 

15 [3] "MPEG-21 Architecture, Scenarios and IPMP Requirements", ISO/IEC JTC1 
SC29/WG11/N5874, July 2003 

The Rights Expression Language (REL) Engine in module 2.1 is the 
component that determines REL authorizations, given an authorization request 
and a set of licenses and root grants. The REL Engine uses the License 
20 Manager to help resolve authorization queries. 

The Digital Item Manager in module 2.2 parses Digital Item 
Declarations within Digital Items. The Digital Item Manager also provides 
access to where the Digital Items are, and creates Digital Item iNstances in 
module 2.3. The Digital Item Manager passes to the License Manager any 
25 Licenses that are embedded within Digital Item Declarations. 

The Digital Item iNstance in module 2.3 represents a Digital Item 
within a Trusted Domain. The Digital Item iNstance contains local metadata 
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about the Digital Item, such as storage location and possibly information about 
content encryption keys. 

The License Manager in module 2.4 supports the REL Engine by 
managing the persistent state of Licenses and their authorization or revocation 
5 status. The License. Manager is also responsible for verifying the integrity of 
Licenses. 

The Condition Processor in module 2.5 selects, evaluates and fulfills 
Conditions, and initiates the execution of authorized Operations (via the DIP 
Processor, generating a Right Exercise) once conditions are satisfied. 

10 The IPMP User Session Manager in module 2.6 orchestrates the 

invocation of Digital Item Operations (via the Condition Evaluator), first 
making sure that proper authorization is obtained (via the REL Engine) and 
that conditions are evaluated (via the Condition Evaluator). 

ARight Exercise in module 2.7 is a record of having exercised a right, Le., 

15 the invocation of a Digital Item Operation. It is maintained by the User 
Session Manager, and is used to associate the fulfillment of Conditions with the 
exercise of Rights. 

The Digital Item Processing Engine in module 2.8 executes Digital Item 
Operations, including Digital Item Methods (DIMs), Digital Item Basic 

20 Operations (DIBOs) in module 2.9, and Digital Item extended Operations 
(DIXOs) in module 2.10. The DIMs are executed by a DIM Engine, the DIXOs 
by a DIXO Engine, and the DIBOs by a DIBO Library. The Digital Item 
Processing Engine updates the User Session State with process state 
information. 

25 The big issue with Figure 2 is that there is no protection information to 

be processed, interpreted and transferred, especially when content is protected 
by several tools and rights is also protected using different tools. There is no 
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clear picture for people to know how the content is protected and how it should 
be processed. 

The second issue with Figure 2 is that the data flow from License 
Manager should not go to REL Engine since the existing REL engine defined in 
5 MPEG-21 REL only processes rights expression. The output from license 
manager could contain the encryption key which is used to decrypt the content 
controlled by an entity which should be IPMP Manager shown in Figure 9. The 
decryption itself can be done in IPMP Tbols, DIP Processor, DIME, or DIBO, or 
DIXO. 

10 The third issue with Figure 2 is that there is no data flow indication to 

indicate where those REL data comes from, for REL Engine to process. Such 
Rights Expression including rights conditions if they are expressed in MPEG-21 
REL format, they could be carried as metadata together with DI in a DID 
container, and processed by DI Manager. DI Manager should be changed into 

15 DID Parser which only parses information by following what DID is defined. 

The better rights and protection is designed based on the two cases. The 
first case is where the existing REL is employed for expressing the 
corresponding rights and conditions and a protection control mechanism is 
defined to take care of content protection including encryption, watermarking, 

20 key management. The second case is where the existing REL is extended by 
adding protection function which could include encryption, watermarking, key 
management, etc. 

Both cases are elaborated in the following sections. 

25 (Content Packaging and Consumption with Separate Rights and Protection) 

As in Figure 3, it is shown on the content packaging side with rights and 
protection scheme. REL in module 3.8 is the existing rights expression language 
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to be used to package the relevant rights with their conditions. Other parts 
through 3.3, 3.4, 3.5, 3.6, 3.7, 3.9, 3.11, and 3.13 are the protection related 
functions. The most important part is in module 3.15, which is the IPMP Control 
Graph. It can be carried in DID container in MPEG-21, but it also can be carried 
5 in other places in diffe rent application domains. 

When the content is needed to transmit via network, normally it will be 
segmented, encrypted and stored as Resource somewhere, and the 
corresponding time-variant key is stored as Key Information in KeyData Holder 
in IPMP Control Graph in module 3.9 directly or indirectly by pointing to a 
10 location. 

For example when the protected content is transmitted over RTP, IPMP 
Control Graph can be carried in SDP (Section Description Protocol), while the 
key information can be carried in the RTP header or as special case for video and 
audio packet as long as there is synchronization among time- variant keys and 
15 the protected video or audio data. 

Module 3.1 is to assign content ID, DII in MPEG-21 could be used here. 
If necessary sub content ID can be used and the protection can be associated 
with this sub content ID if the sub content need to be protected. 

Module 3.2 is to place a flag in IPMP Control Graph to tell if the content 
20 is protected or free. Module 3.3 is to place a flag in IPMP Control Graph to 
indicate if there is watermarking embedded. 

If there is watermarking embedded in the content, module 3.4 will 
assign watermarking (WM) ToolID for the WM tool used for this case, and 
TbolID is then recorded and placed in IPMP Control Graph. The module 3.5 will 
25 create WM Descriptions including watermarking Interface or API related 
information which is placed in IPMP Control Graph. 

Module 3.6 is to determine if the content will be encrypted, and a flag for 



WO 2005/036882 



15 



PCT/JP2003/013120 



"Yes/No" will be placed in IPMP Control Graph in module 3.15. 

Module 3.9 is to assign encryption ToolID for the encryption tool used for 
this case, and ToolID is then recorded and placed in IPMP Control Graph. The 
module 3.7 is to place Key information in KeyData Holder directly in IPMP 
Control Graph, or pointing by the Holder to other location. 

The encryption key can be further encrypted in module 3.11, and 3.13, 
and the key as a license is eventually placed in IPMP Control Graph, REL, DID, 
or somewhere indicated by KeyLocationl. 

Module 3.8 is to create and package rights with the corresponding 
conditions which conforms to the existing REL standard, and this part could be 
modified and edited by distribution agents in the content distribution value 
chain. 

The module 3.10 is to protect the rights metadata by digitally signing 
the rights. Module 3.12 is to assign ToolID for the verification of the digital 
signature, and module 3.14 is to place the Entity_Key in IPMP Control Graph, 
or in DID, or in somewhere indicated by KeyLocation2. 

The detail of module 3.15 is shown in Figure 7 (a) as an example in the 
case of MPEG-21 where XML based approach is used to express IPMP Control 
Graph. ADI (7.2, declared by a DID 7.1) consists of two Items (7.2, 7.3), each of 
which has their identification scheme (7.4, 7.5) with respective attached media 
resource (7.8, 7.9). Module 7.6 shows the IPMP Control Graph mentioned above 
and Module 7.7 gives the actual rights expression (conditions and usage rules) 
finked to the resource. 

Figure 4 shows the Terminal Processing Flow Chart to process 
protection & Packaging Information carried in IPMP Control Graph before a 
protected content could be consumed in module 4.18. 

Module 4.1 is to parse DID and IPMP Control Graph information where 
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DID parser is required only for the case IPMP Control Graph is carried in DID 
in MPEG-21 case. 

In the case of content distribution over RTP network, IPMP Control 
Graph can be retrieved from SDP to obtain rights and protection description 
information except the key information if it is time-variant. 

Module 4.2 is to detect if the content is protected or free. If it is free, it 
will be able to play back by module 4.18 for consumption. Otherwise there are 
three branches to go and check in module 4.3, 4.4, and 4.5, respectively. 

Module 4.3 is to detect if the Rights is encrypted, module 4.4 is to detect 
if the content is encrypted, and module 4.5 is to detect if the content is 
watermarked. 

If the rights is protected, module 4.6 is to invoke the protection tool with 
TbolID and module 4.7 is to check the integrity of the rights using the tool. If the 
integrity is successfully verified in module 4.8, the rights will be sent to module 
4.9 for parsing the rights by RBL Engine which conforms to the existing REL 
standard. 

Module 4.11 is to process the rights and conditions attached to the 
content and store the entitled rights and conditions in a buffer. In module 4.19 
those rights requested by the users are subjected to checking against the rights 
and conditions stored in the buffer. 

If there is license carried in Rights, module 4.10 is to retrieve license 
from License Manager which may be temper resistant (TR) protected. 

If the content is protected and encrypted, module 4.13 is to invoke the 
encryption tool indicated by TbolID carried in IPMP Control Graph, module 4.14 
is to retrieve Key Information, and module 4.12 is to obtaining the key license 
from License Manager. 

License Manager here could be protected by temper resistant technique 
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if it is part of the terminal or somewhere in other places, since it will provide the 
actual license which the decryption engine will use to un-protect the content. 

The encryption tool can be defined as default for most of the terminals to 
use in their implementation, while an IPMP TbolID is provided so that people 
5 can choose other than default encryption tool in their special domain. If the 
platform is allowed to download and use different encryption tool indicated by 
ToolID, it would achieve extensibility, flexibility and renewability at the same 
time we will achieve interoperability across different domains. 

Key Information could be retrieved from different places in the case of 
10 content delivery via various networks. This will depend on where you place key 
information. If you place them in RTP header, you can get them there, while if 
you place them as other packets like video and audio data, you can get them by 
following the same rules applied to video and audio. The time-variant key 
information is required to obtain in the same time when you need to decrypt the 
15 video and audio content. 

Module 4.15 is to decrypt the content with the invoked tool, KeyData, 
and License, then passed to module 4.17 for further processing. 

If the content is detected as watermarked in module 4.5, the 
watermarking tool with IbolID and its description data including interface will 
20 be invoked and prepared in module 4.16 for action which is up to user's request. 

Finally module 4.17 is to exercise the rights which user is requested 
based on the entitled rights & conditions, and act on the un-protected content 
which is the output of module 4.15. 

In Figure 4 Tamper Resistant is used to protect the function of License 
25 Manager to provide license, Rights & Condition Processing to prepare the rights, 
even content decryption for obtaining un-protected content. 

Figure 8 shows a modified IPMP Architecture with REL and IPMP 
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Control Graph separately processed. Compared to the Rights and Protection 
(IPMP Related) functions in Figure 4 and Figure 8, it is clear that there are 
many IPMP related functions missing in the prior art of Figure 2. Only the 
blocks in blue color in Figure 4 which are the module 4.9 for REL Engine, 
5 module 4.10 and 4.12 for License Manager, and module 4.11 for Conditions 
Processing, are introduced in the prior art as shown in Figure 2. Such function 
blocks are module 2.1, module 2.4, and module 2.5 in Figure 2. 

As shown in Figure 8, Module 8.11 is added for parsing and processing 
IPMP Control Graph information, and the corresponding results are passed to 
10 License Manager in module 8.4, REL related data passed to REL Engine in 
module 8.1 after its integrity is checked, and content protection and 
watermarking information passed to DI iNstanace in module 8.3 for further 
processing. 

Decrypting, watermarking, etc. in module 8,12, could be conducted in 
15 module 8.8 if such method is defined in DIME, or in module 8.9 if it is defined as 
one function of DIBO, or in module 8.10 if it is an external function. 

The line 8.14 is shown for the data flow from IPMP Control Graph 
processing module to REL Engine, and the line 8.15 is shown for the data flow 
from IPMP Control Graph processing module to NI iNstance. 
20 The line 8.16 is shown for the data flow from License Manager to the 

un-protecting block in the module 8.12 for issuing a license. 

Module 8.13 is for Event Reporting Engine which is placed in the same 
trusted domain compared to that in Figure 2. 

TR means Temper Resistance module to be used to protect License 
25 Manager operation and Condition Processing Operation. 

Other modules have the similar meaning as explained in Figure 2. 
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(Content Packaging and Consumption with Mixed Rights and Protection) 

In this case, there is no clear boundary between rights and protection, 
and they are mixed. IPMP Control Graph can be considered as REL-IPMP 
Control Graph. 

5 Based on the current MPEG-21 REL or other rights expression language, 

protection of content as well as indicating for how to protect the content is not 
defined. In this case the existing REL has to be extended to support such 
protection signaling. 

As shown in Figure 5 which is based on Figure 3, Module 5.16 is 
10 considered as REL + Extension to support content protection signaling by 
extending the existing REL standard, and module 5.15 is changed into REL- 
IPMP Control Graph. Module 5.8 is the existing REL function. 

Other modules have the same functions as explained above. 
As in Figure 5, it is shown on the content packaging side with rights and 
15 protection scheme. REL in module 5.8 is the existing rights expression language 
to be used to package the relevant rights with their conditions. Other parts 
through 5.3, 5.4, 5.5, 5.6, 5.7, 5.9, 5.11, and 5.13 are the protection related 
functions. The most important part is in module 5.15, which is the REL-IPMP 
Control Graph. It is carried in DID container in MPEG-21, but it also can be 
20 carried in other places when it is used in different application domains. 

When the content is needed to transmit via network, normally it will be 
segmented, encrypted and stored as Resource somewhere, and the 
corresponding time-variant key is stored as Key Information in KeyData Holder 
in REL-IPMP Control Graph in module 5.9 directly or indirectly by pointing to a 
25 location. 

For example when the protected content is transmitted over RTP, REL- 
IPMP Control Graph can be carried in SDP (Section Description Protocol), while 
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the key information can be carried in the RTP header or as special case for video 
and audio packet as long as they are synchronized among time-variant keys and 
the protected video or audio data. 

Module 5.1 is to assign content ID, DII in MPEG-21 could be used here. 
5 Module 5.2 is to place a flag in REL-IPMP Control Graph to tell if the content is 
protected or free. Module 5.3 is to place a flag in REL-IPMP Control Graph to 
indicate if there is watermarking embedded. 

If there is watermarking embedded in the content, module 5.4 will 
assign watermarking (WM) TbolID for the WM tool used for this case, and 
10 TbolID is then recorded and placed in REL-IPMP Control Graph. The module 
5.5 will create WM Descriptions including watermarking Interface or API 
related information which is placed in REL-IPMP Control Graph. 

Module 5.6 is to determine if the content will be encrypted, and a flag foi* 
"Yes/No" will be placed in REL-IPMP Control Graph in module 5.15. 
15 Module 5.9 is to assign encryption IbolID for the encryption tool used for 

this case, and IbolID is then recorded and placed in REL-IPMP Control Graph. 
The module 5.7 is to place Key information in KeyData Holder directly in REL- 
IPMP Control Graph, or pointing by the Holder to other location. 

The encryption key can be further encrypted in module 5.11, and 5.13, 
20 and the key as a license is eventually placed in REL-IPMP Control Graph, REL, 
DID, or somewhere indicated by KeyLocationl. 

Module 5.8 is to create and package rights with the corresponding 
conditions which conforms to the existing REL standard, and this part could be 
modified and edited by distribution agents in the content distribution value 
25 chain. 

The module 5.10 is to protect the rights metadata by digitally signing 
the rights. Module 5.12 is to assign IbolID for the verification of the digital 
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signature, and module 5.14 is to place the Entity_Key in REL-IPMP Control 
Graph, or in DID, or in somewhere indicated by KeyLocation2. 

The detail of module 5.15 is shown in Figure 7 (b) as an example in the 
case of MPEG-21 where XML based approach is used to express REL-IPMP 

5 Control Graph. The figure is similar to Figure 7 (a). It uses REL-IPMP Control 
Graph (7.10) to replace 7.6 and 7.7 modules as shown in Figure 7 (a) but act as 
similar function to represent all rights and protection information. 

It can be seen from the Figure 7 (b) that the REL IPMP extension is 
defined here to contain not only rights expression but also protection 

10 descriptions, and such extension is done on the top of the existing MPEG-21 
REL or other Rights expression language since they are originally defined just 
to express rights, conditions, as well as principles and issuers. The ipmpx shown 
in the XML expression in Figure 7 (b) is the part of the extension of REL for 
protection. 

15 As shown in Figure 6 which is based on Figure 4, Module 6.19 is 

m 

considered as REL + Extension to support content protection as well by the 
extended REL, and module 6.9 is the existing REL engine. Module 6.1 is 
changed into REL-IPMP Control Graph, and Module 6.0 is a separate DID 
parser in the case of MPEG-21. 
20 Other modules are the same functions as explained in the above. 

In Figure 6 it is shown for the Terminal Processing Flow Chart to 
process protection & Packaging Information carried in REL-IPMP Control 
Graph before a protected content could be consumed in module 6.18. 

Module 6.1 is to parse DID and REL-IPMP Control Graph information 
25 where DID parser is required only for the case REL-IPMP Control Graph is 
carried in DID in MPEG-21 case. 

In the case of content distribution over RTP network, REL-IPMP 
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Control Graph can be retrieved from SDP to obtain rights and protection 
description information except the key information if it is time-variant. 

Module 6.2 is to detect if the content is protected or free. If it is free, it 
will be able to play back by module 6.18 for consumption. Otherwise there are 
5 three branches to go and check in module 6.3, 6.4, and 6.5, respectively. 

Module 6.3 is to detect if the Rights is encrypted, module 6.4 is to detect 
if the content is encrypted, and module 6.5 is to detect if the content is 
watermarked. 

If the rights is protected, module 6.6 is to invoke the protection tool with 
10 TbolID and module 6.7 is to check the integrity of the rights using the tool. If the 
integrity is successfully verified in module 6.8, the rights will be sent to module 
6.9 for parsing the rights by REL Engine which conforms to the existing REL 
standard. 

Module 6.11 is to process the rights and conditions attached to the 
15 content and store the entitled rights and conditions in a buffer. In module 6.19 
those rights requested by the users are subjected to checking against the rights 
and conditions stored in the buffer. 

If there is license carried in Rights, module 6.10 is to retrieve license 
from License Manager which may be temper resistant (TR) protected. 
20 If the content is protected and encrypted, module 6.13 is to invoke the 

encryption tool indicated by TbolID carried in REL-IPMP Control Graph, 
module 6.14 is to retrieve Key Information, and module 6.12 is to obtaining the 
key license from License Manager. 

License Manager here could be protected by temper resistant technique 
25 if it is part of the terminal or somewhere in other places, since it will provide the 
actual license which the decryption engine will use to un-protect the content. 

The encryption tool can be defined as default for most of the terminals to 
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use in their implementation, while an IPMP ToolID is provided so that people 
can choose other than default encryption tool in their special domain or case. If 
the platform is allowed to download and use different encryption tool indicated 
by ToolID, it would achieve extensibility, flexibility and renewability at the same 
5 time we will achieve interoperability across different domains. 

Key Information could be retrieved from different places in the case of 
content delivery via various networks. This will depend on where you place key 
information. If you place them in RTP header, you can get them there, while if 
you place them as other packets like video and audio data, you can get them by 
10 following the same rules applied to video and audio. The time-variant key 
information is required to obtain in the same time when you need to decrypt the 
video and audio content. 

Module 6.15 is to decrypting the content with the invoked tool, KeyData, 
and License, then passed to module 6.17 for further processing. 
15 If the content is detected as watermarked in module 6.5, the 

watermarking tool with IbolID and its description data including interface will 
be invoked and prepared in module 6.16 for action which is up to user's request. 

Finally module 6.17 is to exercise the rights which user is requested 
based on the entitled rights & conditions, and act on the un-protected content 
20 which is the output of module 6.15. 

In Figure 6 Tbmper Resistant is used to protect the functioning of 
License Manager to provide license, Rights & Condition Processing to prepare 
the rights, even content decryption for obtaining un-protected content. 

Figure 9 shows for a modified IPMP Architecture with REL-IPMP 
25 Control Graph processed. Compared to the Rights and Protection (IPMP 
Related) functions in Figure 6 and Figure 9, it is clear that there are many 
IPMP related functions missing in the prior art of Figure 2. Only the blocks in 



WO 2005/036882 



24 



PCT/JP2003/013120 



blue color in Figure 6 which are the module 6.9 for REL Engine, module 6.10 
and 6.12 for License Manager, and module 6.11 for Conditions Processing, are 
introduced in the prior art as shown in Figure 2. Such function blocks are 
module 2.1, module 2.4, and module 2.5 in Figure 2. 
5 As shown in Figure 9, Module 9.11 is added for parsing and processing 

IPMP Control Graph information, and the corresponding results are passed to 
License Manager in module 9.4, REL related data passed to REL Engine in 
module 9.1 after its integrity is checked, and content protection and 
watermarking information passed to DI iNstanace in module 9.3 for further 
10 processing. 

Decrypting, watermarking, etc. in module 9,12, could be conducted in 
module 9.8 if such method is defined in DIME, or in module 9.9 if it is defined as 
one function of DIBO, or in module 9.10 if it is an external function. 

The line 9.14 is shown for the data flow from REL-IPMP Control Graph 
15 processing module to REL Engine, and the line 9.15 is shown for the data flow 
from REL-IPMP Control Graph processing module to NI iNstance. 

The line 9.16 is shown for the data flow from License Manager to the 
un-protecting block in the module 9.12 for issuing a license. 

Module 9.13 is for Event Reporting Engine which is placed in the same 
20 trusted domain compared to that in Figure 2. 

TR means Tfemper Resistance module to be used to protect License 
Manager operation and Condition Processing Operation. 

Other modules have the similar meaning as explained in Figure 2. 
In Figure 10, Layout of Rights and Protection in IPMP Control Graph 
25 or REL-IPMP Control Graph is shown, where the content ID, the protected 
objects indicator, the protection flags, and the detail rights and conditions as 
well as the detail protection descriptions are placed and carried in this holder. 
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(Effective of Invention) 

The invention is very effective when content is required to be protected 

with rights and conditions, especially such content can be in any data form and 
5 could be transmitted via various network. 

The invention is effective when such protection is required to associate 

with the protected content via content ID, especially such protection information 

is defined as a set of descriptions attached to the protected content using content 

ID, or DIIinMPEG-21; 
10 The invention is effective when such protection is placed in a generic 

IPMP Control Graph holder or REL-IPMP Control Graph holder, which is clean 

and convenient for content creation, content distribution, as well as content 

consumption, and such holder could be carried in DID in MPEG-21 static file 

format or carried in SDP for RTP transmission. 
15 The invention is effective when each of the protection is indicated by 

ToolID so that both defined IPMP tool and external IPMP Tool can be used for 

flexibility, renewalbility and extensibility. 

INDUSTRIAL APPLICABILITY 

20 The present invention relates to Digital Rights Management (DRM) or 

Intellectual Property Management and Protection (IPMP) for a generic digital 
content, especially relates to the protection and management of a digital content 
independent of any data format. The invention is very effective when content is 
required to be protected with rights and conditions, especially such content can 

25 be in any data form and could be transmitted via various network. 



WO 2005/036882 



26 



PCT/JP2003/013120 



CLAIMS 

1. A Methods of Digital Content Protection with Digital Rights 
Expression, comprising the following steps of: 

Parsing a digital content description, especially parsing a DID (Digital 
5 Item Declaration) in MPEG-21 scope; 

Retrieving a digital content identifier (content ID) which is used to 
identify the said digital content, especially a DII in MPEG-21 scope, or sub 
content identifier; 

Detecting a rights and protection description holder which contains 
10 rights and protection information applied to the said digital content with the 
corresponding content ID, and here the holder called IPMP (Intellectual 
Property Management and Protection) Control Graph Holder or REL (Rights 
Expression Language)-IPMP Control Graph holder; 

Retrieving a flag from the said holder which indicates if the said content 
15 is protected or belongs to free content; 

Processing the description information carried in the said IPMP Control 
Graph or REL-IPMP Control Graph; 

Checking if rights descriptions or other metadata description is digital 
signed by retrieving a flag which is attached to the said rights or other 
20 metadata; and if it is signed, preparing the corresponding digital signing tool 
which is indicated by TbolID; 

Retrieving a key license from a protected License Manager; 

Checking the integrity of the said rights or metadata using the said 
digital signing tool; 

25 Parsing the said rights with their conditions following the rules which is 

pre-defined, especially following REL rules which is defined in MPEG-21 scope, 
and storing the said entitled rights and conditions in a buffer for future 
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checking; 

Checking if the said content is encrypted by retrieving a flag which is 
attached to the said content; and if it is encrypted, preparing the corresponding 
encryption tool which is indicated by TbolID; 
5 Un-protecting the said encrypted content using the said encryption tool 

with the said TbolID, and other information; 

Checking if the said content is watermarked by retrieving a flag which is 
attached to the said content; and if it is watermarked, preparing the 
corresponding watermarking tool which is indicated by TbolID for further 
10 action; 

Processing user's request against the said entitled rights and conditions 
stored in the buffer; 

Exercising the rights requested by the said user if it is entitled, and 
Acting on the said un-protected content for playing, rendering, recording, 
15 modifying, deleting, adapting, etc. 



2. A Methods of Digital Content Protection with Digital Rights 
Expression, whereas Un-protecting the said encrypted content using 
the said encryption tool with the said ToolID, and other information 
20 in claim 1, further comprising the following steps of: 

Retrieving the key information from KeyData holder in the said IPMP 
Control Graph or REL-IPMP Control Graph directly or the location 
pointed by a pointer which is placed in IPMP Control Graph or REL- 
IPMP Control Graph, and 
25 Retrieving a key license from a protected License Manager. 
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3. The Methods of Digital Content Protection with Digital Rights 
Expression, whereas Retrieving a key license from a protected 
License Manager in claim 2, further comprising the following steps 
of: 

5 Protecting the said Licence Manager using Temper Resistant approach, 

and 

Protecting the buffer which is used to store the said retrieved and 
generated key license, and the said buffer could be Temper Resistant 
protected. 

10 

4. The Methods of Digital Content Protection with Digital Rights 
Expression, whereas Parsing the said rights with their conditions 
following the rules which is pre-defined, especially following REL 
rules which is defined in MPEG-21 scope, and storing the said 

15 entitled rights and conditions in a buffer for future checking in claim 

1, further comprising the following steps of: 
Protecting the said Rights Parser or part of the said Rights Parser using 
Temper Resistant approach or other approach, and 

Protecting the said buffer using Temper Resistant approach or other 
20 approaches. 

5. The Methods of Digital Content Protection with Digital Rights 
Expression, whereas Checking if rights descriptions or other 
metadata description is digital signed by retrieving a flag which is 

25 attached to the said rights or other metadata; and if it is signed, 

preparing the corresponding digital signing tool which is indicated by 
ToolID in claim 1, here the other metadata description could be DIA 
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(Digital Item Adaptation) in MPEG-21 scope, or other types of 
metadata, which could be protected and the protection description 
information is placed in the said IPMP Control Graph or REL-IPMP 
Control Graph. 

5 

6. The Methods of Digital Content Protection with Digital Rights 
Expression whereas in claim 1 encryption and decryption could be 
done using a defined tool as default with a defined ToolID in certain 
application domain, digital signing could be done using a defined tool 
10 as default with a defined ToolID in certain application domain, and 

other protection such as watermarking could be done by defining an 
interface or API to achieve flexibility. 



7. The Methods of Digital Content Protection with Digital Rights 
15 Expression whereas in claim 1 the operation of un-protecting a 

content could be done in DIP (Digital Item Processing) Engine as 
defined in DIME (Digital Item Method Engine), or DIBO (Digital 
Item Base Operation), or DIXO (Digital Item extended Operation). 

20 8. The Methods of Digital Content Protection with Digital Rights 

Expression whereas in claim 1 the said REL-IPMP Control Graph 
means to extend the existing REL of MPEG-21 or other rights 
expression language to contain protection description information, 
where IPMPX is defined as the flag used to represent the extension 

25 part of protection from the existing REL. 
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FIG. 7a 
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FIG. 7b 
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FIG. 8 
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FIG. 10 



-► Conten t id 



REL Rights _ 
Indicator 

DIA Metadata 
Indicator 

Other Metadata 
Indicator 



Protected 
Flag 

Protected 
FJag 

Protected 
Flag 



-► R ights 
Descriptions 



Protectio n 
Description 

Protection 
Des c ription 

Protection 
Description 




Content 
Location 



Content Protected 
Flag 



Protection 
Description . 

ToollD KeyData License 



& m m, (mm) 



WO 2005/036882 PCT/JP2003/013120 



12/12 

Reference numerals in the drawings 

301 Module to assign content ID, Dll in MPEG-21 

302 Module to place a flag in IPMP Control Graph to tell if the content is protected or free 

303 Module to place a flag in IPMP Control Graph to indicate tell if there is watermarking 
embedded 



INTERNATIONAL SEARCH REPORT 



A. CLASSIFICATION OF SUBJECT MATTER 

IPC 7 H04N7/24 H04N5/00 



According to International Patent Classification (IPC) or to both national classification and IPC 



tatlonal Application No 

/JP 03/13120 



B. FIELDS SEARCHED 



Minimum documentation searched (classification system followed by classification symbols* 

IPC 7 H04N 



Documentation searched other than minimum documentation to the extent that such documents are Included in the fields searched 



Electronic data base consulted during the international search (name of data base and, where practical, search terms used) 

EPO-Internal , WPI Data, PAJ 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category • Citation of document with Indication, where appropriate, of the relevant passages 



Relevant to claim No. 



A 



WO 03/075575 A (SENOH TAKANORI ; 
MATSUSHITA ELECTRIC IND CO LTD (JP); 
MING (SG); HU) 

12 September 2003 (2003-09-12) 
abstract 



JI 



1-8 



page 


5, 


line 12 - 


■ line 21 






page 


11, 


line 


20 


- page 13, 


line 


2 


page 


14, 


line 


10 


- line 14 






page 


15, 


line 


6 - 


• line 16 






page 


17, 


line 


17 


- page 18, 


line 


3 


page 


19, 


line 


12 








page 


20, 


line 


5 








page 


22, 


line 


7 - 


■ line 13 






page 


23, 


line 


21 


- page 25, 


line 


7 


page 


29, 


line 


13 


- line 19 






page 


33, 


line 


7 - 


• page 34, 


line 


11 



-/-- 



m 



Further documents are listed In the continuation of box C. 



10 



Patent family members are listed In annex. 



° Special categories of cited documents : 

■A" document defining the general state of the art which Is not 

considered to be of particular relevance 
•E' earner document but published on or after the international 

filing date 

•L' document which may throw doubts on priority dalm(s) or 
which is cited to establish the publication date of another 
citation or other special reason (as specified) 

"O* document referring to an oral disclosure, use, exhibition or 
other means 

"P* document published prior to the international filing date but 
later than the priority date claimed 



T later document published after the International filing date 
or priority date and not In conflict with the application but 
cited to understand the principle or theory underlying the 
Invention 

'X* document of particular relevance; the claimed invention 
cannot be considered novel or cannot be considered to 
involve an inventive step when the document Is taken alone 

•Y" document of particular relevance; the claimed invention 

cannot be considered to invotve an Inventive step when the 
document Is combined wllh one or more other such docu- 
ments, such combination being obvious to a person skilled 
in the art 

document member of the same patent family 



Date of the actual completion of the International search 

16 June 2004 



Date of mailing of the international search report 

29/06/2004 



Name and mailing address of the ISA 

European Patent Office, P.B. 5818 Patentlaan 2 
NL-2260HVRIlSwnk 
Tel. (+31-70) 340-2040, Tx. 31 651 epo nl, 
Fax (+31-70)340-3016 



Authorized officer 



Fantlni , F 



Form PCT/ISA/210 (second sheet) (January 2004) 



page 1 of 2 



INTERNATIONAL SEARCH REPORT 



Inumational Application No 

W/JP 03/13120 



C(Contlnuatlon) DOCUMENTS CONSIDERED TO BE RELEVANT 



Category • Citation of document, with Indication, where appropriate, of the relevant passages 



Relevant to claim No. 



WO 03/039155 A (SENOH TAKANORI ; 
MATSUSHITA ELECTRIC IND CO LTD (OP); JI 
MING (SG); TA) 8 May 2003 (2003-05-08) 
page 7, line 20 - page 8, line 2 

J. H. KIM, S. 0. HWANG, K. S. Y00N, C. S. 
PARK: "MPEG-21 IPMP"' Online! 
14 September 2002 (2002-09-14), 
XP002284699 

Retrieved from the Internet: 

URL : http : //charybd 1 s . mi t . csu . edu . au/{manto 

lov/CD/ICITA2002/papers/177-21.pdf> 

the whole document 

W0 03/067893 A (SENOH TAKANORI ; 
MATSUSHITA ELECTRIC IND CO LTD (OP); 01 
MING (SG); HU) 14 August 2003 (2003-08-14) 
abstract 

page 4, line 11 - page 5, line 4 
page 6, line 14 - Hne 21 
page 15, line 3 - line 20 



1-8 



1-8 



1-8 



EP 1 079 627 A (CANON KK) 
28 February 2001 (2001-02-28) 
column 13, line 40 - line 58 
column 15, line 35 - line 43 



1-8 



Form PCTY1SA/210 (continuation of second sheet) (Januaiy 2004) 



page 2 of 2 



INTERNATIONAL SEARCH REPORT 

Information on patent family members 



Patent document 
cited in search report 



Publication 
date 



In^rnational Application No 

Wr/JP 03/13120 



Patent family 
member(s) 



Publication 
date 



WO 03075575 



12-09-2003 



EP 1397919 Al 

WO 03075575 Al 
JP 2004129197 A 



17-03-2004 
12-09-2003 
22-04-2004 



WO 


03039155 


A 


08-05- 


-2003 


WO 


03039155 A2 


08-05-2003 












JP 


2004005365 A 


08-01-2004 


WO 


03067893 


A 


14-08- 


-2003 


WO 


03067893 Al 


14-08-2003 












JP 


2004038924 A 


05-02-2004 


EP 


1079627 


A 


28-02- 


-2001 


JP 


2001069457 A 


16-03-2001 












JP 


2001078007 A 


23-03-2001 












JP 


2001203683 A 


27-07-2001 












EP 


1079627 Al 


28-02-2001 



Form PCT/1SA/21 0 (patent family annex) (January 2004) 



